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The use of animated agents in 
e-learning environments: an 
exploratory, interpretive case study 

A. Kamil Mahmood and Elaine Ferneley* 

University of Salford, UK 


There is increasing interest in the use of animated agents in e-learning environments. However, 
empirical investigations of their use in online education are limited. Our aim is to provide an empir- 
ically based framework for the development and evaluation of animated agents in e-learning 
environments. Findings suggest a number of challenges, including the multiple dialogue models 
that animated agents will need to accommodate, the diverse range of roles that pedagogical 
animated agents can usefully support, the dichotomous relationship that emerges between these 
roles and that of the lecturer, and student perception of the degree of autonomy that can be afforded 
to animated agents. 


Introduction 

A new educational paradigm is emerging: face-to-face interaction with intelligent 
pedagogical animated agents in interactive learning environments that are capable of 
performing tasks and achieving goals individually and in collaboration with human and 
non-human agents. Currently, significant work has been undertaken on incorporating 
conversational behaviours of emotion, personality, conversational performatives and 
functions into such agents (Cassell et al., 2000; Ruttkay & Pelachaud, 2004). 
However, little empirical work exists on their utility from the end user perspective. 
That which does exist, while insightful, focuses on the expressiveness of animated 
agents rather than the functionality that end users may require from them (Lester et al . , 
1997). Using an interpretive case study approach, this research has developed a frame- 
work for pedagogical animated agent evaluation that focuses on end-user-defined 
functional requirements. The research participants were early adopters of animated 
agent technologies within e-leaming environments at a Malaysian university. 
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Background 

There is a growing body of literature relating to e-learning including research on the 
adoption of courseware (Russell, 1999), learner experiences with e-leaming (Kara & 
Kling, 1999; Johnson et al., 2000a) and the comparison of the critical attributes of 
traditional and e-learning environments (Blignauta & Trollipa, 2003). Increasingly 
sophisticated features are being conceptualised and incorporated in e-leaming 
systems such as intelligent agents (Lester et al., 2000; Schroeder, 2002), adaptive 
hypermedia (Ng et al., 2002) and game-based learning (Foreman, 2004) that aim to 
make e-learning friendly and engaging for both instructors and learners. 

The inclusion of intelligent agent technology in e-leaming environments has been 
of growing interest (Conole, 2002; Logan et al., 2002; Songa et al., 2004). Intelligent 
agents are computer programs that can act autonomously and perform tasks intelli- 
gently with consideration given to context and user preferences. They support the 
user in carrying out computerised tasks such as information searching, meeting 
scheduling or online purchasing (Maes, 1994). Advances in computer graphics have 
made the embodiment of intelligent agents (Imown as animated agents) possible 
(Schroeder, 2002) and e-learning researchers have proposed the utilisation of 
animated agents to reduce the overheads involved in delivering online education, to 
facilitate dynamic monitoring of course activities and delegation of certain instructor 
tasks to pedagogical animated agents Qohnson et al., 2000b; Conole, 2002; Jafari, 
2002; Songa et al., 2004). The expectation is that learners will be able to interact visu- 
ally with pedagogical animated agents who will be able to exhibit emotive behaviour 
and expressions and simulate continuous instructor presence in e-leaming environ- 
ments. This notion is in line with the futuristic vision of the Peripatetic Electronic 
Teacher, as coined by Squires (1999), the freelance educator providing education 
virtually on demand with their routine tasks supported or delivered by agent-based 
technology that will be required to operate in four key roles or ‘'presences’-, as a peda- 
gogical presence, in a managerial role as a professional (usually in some peer group 
setting) or in a commercial role as a personal marketeer. In the 1990s animated agents 
were developed to support students in the use of educational programs; for example, 
Herman the Bug (Lester et al., 1999), Steve — Soar Training Expert for Virtual 
Environments (Rickel & Johnson, 1999) and Adele — ^Agent for Distance Learning 
(Shaw et al., 1999). However, there is now a sense of urgency in identifying how to 
incorporate pedagogical agent technology appropriately in e-leaming environments. 

While a utopian vision may exist of learners interacting seamlessly with pedagogi- 
cal animated agents, in practice animation can be irritating and empirical investiga- 
tions have revealed diverse results in a continuum from animated agents being 
helpful to them being distracting and hampering the learning process (Dehn & 
Mulken, 2000; Ruttkay & Pelachaud, 2004). Additionally, there are concerns 
regarding their pedagogical value, with no clear evidence that they can enhance 
communication or understanding (Dehn, 2000; Moundridou & Virvou, 2002). 
Hence their contribution to the efficiency and effectiveness of learning continues to 
be an open research agenda. 
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Therefore, this paper can be seen as a response to the question: ‘what kind of 
animated agent used in what land of domain influences what aspects of the user’s 
attitudes or performance?’ (Dehn & Mulken, 2000, p. 19). It will involve examin- 
ing the application of pedagogical animated agents in e-leaming environments. 
Since the question has multiple components, this paper investigates only the final 
portion, providing additional insights into learners’ perceptions of the utility of 
animated agents in e-learning environments. The key result of the research is a 
conceptual design and evaluation framework for the incorporation of animated 
agents in e-leaming environments. 


Methodology 

Examining the use of pedagogical animated agents requires investigation from multi- 
ple stakeholder perspectives (e.g. lecturers, undergraduate and postgraduate 
students) and an understanding of the social and contextual relationships that 
influence the stakeholders — there can be no single explanation of success. Our epis- 
temological assumptions are that no individual account of social reality can be proven 
correct. Therefore, as the fundamental nature of this study is concerned with inter- 
preting human action and perceptions, the researchers drew on an interpretive and 
qualitative research tradition in order to explore and understand the perspectives of 
the various stakeholders and the historical and socially situated contexts in which they 
reside (Walsham, 1995; Klein & Myers, 1999). The usual approach to gaining such 
understanding (or ‘rich insight’), particularly where the phenomena cannot be stud- 
ied outside of the context in which it occurs, is an observation-based case study 
approach (Walsham, 1993; Yin, 1994). 

The case study was conducted in the Information Systems department at a private 
university in Malaysia. The department has made extensive use of e-leaming facili- 
ties, offering as part of its degree programme full online modules and blended learn- 
ing modules since 2000. Since the adoption of e-learning facilities, staff have 
incorporated a variety of animated agents into their online materials, including 
animated instructional agents developed using Microsoft AgenT'^ (Microsoft, 2006), 
the demonstration and problem-solving animated agents Steve and Adele (Rickel & 
Johnson, 1999; Shaw et al., 1999) and information search and retrieval agents such 
as Gossip (Tryllian, 2006). Case study data were collected by the first author who is 
an academic member of staff at the institution and a native Malaysian, speaking the 
language of the participants. This ‘insider’ perspective expedited data collection and 
enabled a deeper understanding of the issues raised. The research design process was 
executed in phases (Yin, 2003). 


First phase: planning and preparation 

In undertaldng qualitative research, there is more than one way in which it may be 
performed, as Markus underlined: 
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If we who claim to understand qualitative research argue that there is only one right way 
to do it, we are deliberately blinding ourselves to many interesting and important research 
questions and phenomena. (1997, p. 15) 

In this study the planning and preparation phase began with a detailed literature 

review. Three seed research questions were explored: 

• What roles can animated agents effectively assume in e-leaming environments? 

• When, how and for whom would those roles be initiated? 

• Do social and cultural factors influence the interaction between the users and 
animated agents? 

A conceptual framework was developed, founded on the research questions (Miles & 

Huberman, 1994). This framework acted as the stimuli for data collection. Briefly the 

framework consisted of the following: 

• Learner characteristics — including, learning style (visual, auditory or kinaestheticj 
Sabry & Baldwin, 2003); system experience (Holscherl & Strubel, 2000), prior 
knowledge and spatial ability (Mayer & Gallini, 1990) and occupational experi- 
ence (Durling et al., 1996). 

• Technology — the various technologies and mediums used to deliver and commu- 
nicate with the learners, including speech, text, sound and animated agent 
expression. 

• Learning environment — physical attributes including monitor size, microphone 
and speaker position and room layoutj and virtual attributes including look and feel 
of the applications, courseware content and structure. 

• Culture — the users’ enacted system of beliefs, symbols and behaviours^ and collec- 
tive programming of the mind such as the learning of ideas, habits, attitudes, 
customs and traditions (Hofstede, 1991). 


Second phase: fieldwork 

Data were collected over a 1 2-month period using participant observation and indi- 
vidual and group interviews. All participants were from the Malaysian university’s 
Information Systems department. Twenty-six interviewees were involved in 21 
separate sessions: six individual lecturer interviews, four individual postgraduate 
interviews, one group postgraduate interview (two participants), eight individual 
undergraduate interviews and two group undergraduate interviews (three partici- 
pants in each group interview) . While the majority of interviews were individual, three 
small group interview sessions were undertaken with students to explore whether 
different issues emerged, although no signiflcantly different issues did. Interviews 
began with more generic questioning allowing users to express their opinions on the 
use of pedagogical animated agents before moving to more speciflc issues to ensure 
that data from each interview covered similar areas thus allowing cross-respondent 
comparison. Data collection ceased at the point of data saturation (Miles & 
Huberman, 1994). Simulated propositions were presented to the interviewees in the 
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form of a series of small video clips from the Star Trek television series that showed 
possible future applications for animated agents. The aim was to allow discussion of 
possible future scenarios but to ‘bound’ the discussion and give some direction to 
respondents to allow cross-respondent comparison. The videos included examples of 
animated agents on Star Trek’s holodeck and embedded agents in Star Trek’s engi- 
neering department. 

The sampling approach for choosing the interviewees was driven by the emerging 
themes. The aim was not to capture the representativeness of the population, but 
rather to continuously solicit and analyse the ‘hows’ of meaning and the ‘whats’ of 
interviewees’ experiences until the findings were either being reinforced or emergent 
issues were saturated. During both the observation sessions and interviews the 
researcher took field notes, highlighting potential emergent themes. All interviews 
were video-recorded and conducted in the language and style of expression that was 
most convenient for the interviewees to convey meaning, which at times was a mix of 
English and Malay. Interviews lasted between 50 minutes and 2 hours 20 minutes. 
Table 1 provides an overview of the structure, format and issues covered in the inter- 
views, although this is only indicative as an open rather than closed questioning style 
was used. 


Third and fourth phases: summarising and aggregating the data 

Full transcripts were made of all interview sessions, and summaries were made of all 
observational sessions. Using an inductive analytical approach an initial coding 
scheme derived from the initial, literature-based, conceptual framework was further 
explored and refined by application of the case study data, providing a ‘behavioural 
description’ of the use of pedagogical animated agents (Miles & Huberman, 1994). 
As coding progressed a number of themes began to emerge; namely, the various roles 
that pedagogical animated agents could assume, the various effects that pedagogical 
animated agents have on users and context and cultural factors that may influence 
user interaction. Because qualitative data analysis is an open and iterative process, 
applying the initial coding scheme to the case studies resulted in the emergence or 
induction of a richer coding set as the initial codes were ‘extended’, ‘filled in’, 
‘bridged’ and ‘surfaced’ (Lincoln & Cuba, 1985). For illustration, the ‘roles’ cluster 
evolved to include a set of subclusters including the roles of personal assistant, 
demonstrator, counsellor and peer (see also Table 2). Validation of the coding 
scheme was undertaken by gaining feedback from interviewees and the co-researcher 
and by applying the principles of the hermeneutic cycle (Klein & Myers, 1999). 
Qualitative data analysis software was used to assist in the coding process. These 
subclusters are supported by various logical chains of evidence matrices. For illustra- 
tion, an extract from the role-ordered matrix that shows a subset of the emergent roles 
is presented in Table 2. The role-order matrix was created by extracting the various 
requirements and issues that each type of respondent identified as factors in the use 
of pedagogical animated agents. For illustration and due to space limitations only one 
or two requirements/issues are presented for each cell within the role-order matrix. 


Table 1 . Overview of the structure, format and issues covered in the interviews 
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Results 

The key findings abstracted from the matrices were: 


Animated agent roles 

The data analysis revealed a range of potential roles for animated agents including 
personal tutor, personal assistant, mentor, academic counsellor, lecturer, entertainer, 
buddy and peer. The instantiation of these roles was viewed very differently by 
academic and student respondents. Considering Squire’s (1999, p. 54) four 'presence’ 
domains, broadly, students focused on animated agents providing a pedagogical pres- 
ence, while academics expected animated agents to assume a managerial stance. 
Neither user group considered Squires’s professional or commercial perspectives — 
for example, animated agents as representatives at academic meetings or animated 
agents in some form of marketing or sales role. 


Lecture perspective. Lecturers were willing to, and many did, delegate laborious, 
routine ‘low-risk’ tasks to animated agents, including searching for academic 
and administrative information, managing appointments and replying to non-crucial 
e-mails. One of the lecturers stated: 

If the agents can do half of my routine tasks, that would be nice ... as an academic, I can 
spend that valuable spare time for more productive activities. I don’t mind if those routine 
tasks are handled by agents. 

However, they identified the major barrier to adoption included the time overhead 
required to gain familiarity with the technology, the training curve required by agent 
technologies and past unsatisfactory results from using agents 

For more advanced roles it was recognised that a more constructivist, learner- 
centred approach to education would benefit students and that pedagogical animated 
agents can operate as a ‘guide-on-the-side’. For example, animated agents explaining 
subject matter in an alternative way, rephrasing the material and providing alternative 
diagrams, animations and texts to support the student’s learning style had been 
trialled and were discussed. A lecturer stated: 

The agents should be able to explain in different ways ... the agent should be able to 
capture, rephrase and reinforce answers or questions. 

Concern was expressed that learners do not trust the technology, that their atten- 
tion may wane, that animated agents may tarnish the image of the lecturer, especially 
when the agents have a strong association with the lecturer, that possible negative 
effects on lecturer-student relationships may ensue and that animated agents may 
pose a potential threat to teaching careers. Additionally, interviewees were concerned 
that animated agents were a passing fad and they had neither the time nor the ability 
to change their teaching styles to fully incorporate them into their online courses, 
especially if animated agents were required to present an alternative pedagogical 
approach to the material being delivered. 
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Student perspective. Postgraduates were not interested in animated agent aesthetics. 
For example: 

If it fails to deliver I am not going to use it. Look is secondary — it is the practicality, speed 
and quality of response that count. 

They focused on the possibility of advanced animated agent roles such as advisor or 
mentor, as stated by one respondent, and typical of others: 

My current problem as research student is how to understand what I read . . . interactive 
software could help, mayhe in terms of visualisations instead of just reading the literature. 

Potential animated agent roles were advanced with context specific requirements 
that are at the boundaries of current technology. They proposed a mediating role of 
‘team buddy’, providing support for groups of research postgraduates and stimulating 
collaborative research or learning. 

Undergraduates focused on more informal animated agent roles to help them over- 
come feelings such as shyness and inferiority, by engaging in a more open, informal 
environment, as stated by one respondent: 

Agents can actually help the student in a ‘virtual way’, there is no connection between 
lecturer and student, ... can get the answers or help outside the classroom. 

Undergraduates also emphasised the need for 24-hour service, for repeated one-to- 
one interaction on the same topic within a confidential environment and interaction 
at the intellectual level of the undergraduate — a personalised user profile. This concept 
of service and personalisation was linked to interest retention, and analogies between 
gaming software and e-leaming environments were repeatedly made; for example: 

I think if the animated software agents play their roles right, they can help to make the 
courseware as interesting as the games software, and . . . retain the interest of the learner. 

For example, we can revise the assignments to look more like games software rather than 
like academic problems. 

Despite undergraduates’ enthusiasm for animated agents there was concern that 
such technology could not sustain high motivation and engagement in learning. Such 
motivational activity was regarded as a human process. 


Social and cultural influences 

As social and cultural influences are an intrinsic part of learning, the issue of cust- 
omisability that allows factors such as language, looks and communication styles to 
be incorporated into an animated agents was raised. Culturally, the Malaysian 
respondents focused on the inclusion of Asian characteristics into animated agent 
designs for reasons of ‘national pride’. Also, with respect to the Malaysian context, 
there was a reluctance to express dissent, to query or be critical of academic author- 
ity. The respondents saw the introduction of pedagogical animated agents as an 
opportunity to act in a less reserved fashion and adopt a more questioning, critical 
style of interaction. However, despite cultural factors being mentioned, it was 
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repeatedly stated that quality of interaction with animated agents was more relevant 
than cultural identity. 

Socially, interviewees focused on animated agents being able to personalise their 
interaction with learners dependent on a user’s profile, considering attributes such as 
past interactions and learner style with the aim of improving interaction and simpli- 
fying steps towards understanding the subject under study. Interviewees also 
expressed a desire for animated agents that used informal, friendly language, commu- 
nicating at the intellectual level of the user and taking the role of tutor, counsellor or 
buddy rather than the more authoritarian lecturer stance. The role of entertainer was 
also mentioned — an animated agent that can identify when the user is bored, can 
distract or even ‘crack an academic joke’: the overarching desire was for a high level 
of context dependent personalisation. 

Animated agent design framework 

The multiple academic and student perspectives were summarised in a framework for 
animated agent design in e-leaming environments. Figure 1 illustrates this framework 
graphically using the qualitative data analysis NVivo Modeller software. 
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Figure 1 . Animated agent design framework 


164 A. K. Mahmood and E. Femeley 


Discussion 

Pedagogical animated agents have many similarities with other work in intelligent 
tutoring systems and interactive e-leaming environments including reasoning capa- 
bilities, user profiling, mechanisms for evolving with the user, pedagogical approach 
and the maintenance of a logical dialogue with the user (Cassell et al., 2000', Ruttkay 
& Pelachaud, 2004). However, previous research has drawn on empirical studies in 
other domains to identify the utility of pedagogical animated agents in e-learning 
environments. This in-depth case study research has been conducted with experi- 
enced e-leaming users who had used pedagogical animated agents previously to 
develop a framework of key requirements for pedagogical animated agents in e-leam- 
ing environments. 


Animated agent roles 

A range of roles were identified that a pedagogical animated agent could assume: 
Tutor, Personal Assistant, Lecturer, Buddy, Entertainer and Counsellor. As with any 
autonomous agent, the agent must be able to react dynamically with its environment 
while maintaining a coherent dialogue with the user. The possibility of assuming 
multiple roles and, at the same time being embodied, raises particular challenges for 
the animated agent designer. 


Dialogue context. The instructional goals may vary dependent on the animated 
agent’s role and the stage in the dialogue. A dialogue model is required that considers 
both the ultimate and immediate conversational goals. In a dynamic environment the 
design of the animated agent to allow seamless transfer between roles while maintain- 
ing coherence will be particularly challenging. The designer may consider alternative 
embodiments allowing a range of non-verbal behaviours depending on role rather 
than attempting to combine roles into a single animated agent. 


Visual context. A key difficulty in animated agent design is the design of suitable 
non-verbal behaviours; for example, appropriate changes in facial expression. In this 
study respondents repeatedly stated that the visual representation was secondary to 
service quality, and therefore e-leaming designers may be advised to utilise simplistic 
embodiments until the animated agent designers have further addressed issues of 
believability. 


Pedagogical context. Dependent on the animated agent’s role, the pedagogical intent 
will vary. While the capture of appropriate dialogue to support the more formal 
Lecturer or Tutor roles may be time consuming, it is unlikely that there would be 
ambiguity in meaning. However, if alternative pedagogical approaches and peer-type 
support is to be included then there is a key question regarding how the appropriate, 
alternative pedagogical structures will be captured. In order to support multiple roles. 
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multiple pedagogical perspectives (presumably captured from multiple academics) 
would be required. This study highlighted a number of concerns from academics 
regarding engaging with such technology. If said technology presents pedagogical 
views that do not concur with the academic user’s perspective, the likelihood of adop- 
tion is significantly reduced. There is a dichotomy here between the utility of provid- 
ing multiple animated agent roles and the likelihood of adoption by academics. 


Animated agent service quality and usefulness 

Unsurprisingly the service quality and usefulness of animated agents emerged as a key 
theme, specifically as respondents had previously experienced animated agents that 
were distracting and annoying. 


Presence context. Animated agents as a distraction or annoyance emerged, particu- 
larly if inappropriate behaviours were assigned to them. The respondents, both 
educators and students, emphasised the desire to be in control of the animated 
agents, to be able to determine when to engage with them. Yet, from the educator and 
student perspectives the desire for control was pedagogically contradictory. Lecturers 
expressed a desire for highly controlled agents that they could program in detail to 
structure the learning process. Students similarly wished to control agent interaction 
and determine the role an animated agent should adopt dependent on context (e.g. 
peer, buddy or tutor). A key challenge for future designers is how to accommodate 
these two views of where control should be located. 


Profile context. For multiple, context-dependent roles and behaviours to occur a user 
profiling approach would appear appropriate. A user’s learning style, career history, 
interests, attitudes and culture are all unique making their learning experience a 
personal journey. To incorporate such attributes into a personalised agent has long 
been an aim in agent design (Maes, 1994) and is a vision that has been incorporated 
into Squires’ notion of the Peripatetic Electronic Teacher (Squires, 1999). However, 
this utopian vision is still a long way from realisation, and pedagogical animated agent 
developers need to face the challenge of integrating user profiling and agent technol- 
ogy while both fields are still in their infancy. 


Conclusions 

Learners are being confronted with an ever-increasing quantity of information while 
at the same time the distance between themselves and their educator becomes either 
physically or conceptually greater. This ‘gap’ has been partially filled by e-learning 
environments. More recent research has moved towards the use of pedagogical 
animated agents, focusing on conversational behaviours and design features (Cassell 
et al, 2000; Johnson et al., 2000b; Ruttkay & Pelachaud, 2004). However, in order 
to achieve Squires’s futuristic vision of the Peripatetic Electronic Teacher, a clearer 
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understanding of end users’ expectations of pedagogical animated agents is required 
Qohnson et al., 2000a; Conole, 2002; Logan et al., 2002). 

The research presented here has employed an in-depth interpretive case study 
approach to explore the issues surrounding the use of pedagogical animated agents in 
e-learning environments. The case study site has employed animated agents and e- 
leaming as part of their educational delivery strategy for over five years. While the 
latest research on animated agents in e-leaming environments has focused on design 
issues of believability and deictic behaviour, the participants in this research were 
more concerned with exploiting the classic characteristics of intelligent agents, such 
as autonomy, proactiveness and social ability. The participants already had experi- 
ence of pedagogical animated agents and repeatedly expressed concern that interface 
issues did not dominate to the expense of functional requirements. The research 
identified three key themes for pedagogical animated agent development: the instan- 
tiation of multiple roles, quality of service and a user-profile-based approach. Taking 
these three themes a number of issues arise: 

• Potential conflicting roles of the various animated agents. As a learner begins to 
interact with multiple animated agents, how can a consistent dialogue model be 
maintained? 

• Population of animated agents. In order to provide the quality of service, multiple 
experts may be required to work collaboratively to populate the various animated 
agent roles; for example, to provide alternative pedagogical models on specific 
topics. This has significant set-up cost implications. Dichotomously, lecturers 
wished to maintain personalised animated agents that presented their material in a 
particular format and style. 

• Quality assurance procedures. If pedagogical animated agents are ultimately to be 
autonomous, proactive and socially agile then clearly defined quality assurance 
procedures need to be put in place to ensure that the pedagogical integrity of the 
animated agents is maintained. 

• Development of user profiles including learning styles, career background and 
interests in order to develop a personalised presentation style. For example, in 
order to tailor learning to an individual’s requirements it may be useful to know 
whether they are typically classed as a visual, auditory or kinestatic learner. Yet, 
how would a user profile be captured? For example, a learner will not necessarily 
know which learning style is most appropriate for their needs while a pedagogical 
animated agent will need to know how to present information to best effect depen- 
dent on the particular learner. 

The issues raised highlight the speculative nature of this field and are based on a 
single case study. However, the case study respondents had been exposed to both e- 
leaming environments and pedagogical animated agents for some five years. While 
the implications suggested here might not be applicable to the individuals outside this 
sample (Lee & Baskerville, 2003), the findings and suggestions, as in any case study, 
may be useful for further study and act as a foundation for further longitudinal 
research. 
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